Operating sub-pixel rendering filters in a display system

ABSTRACT

A method of operating spatial sampling filters comprises detecting subpixel rendered areas, turning on a first set of spatial sampling filters for the subpixel rendered areas in response to detecting said subpixel rendered areas, detecting non-subpixel rendered areas; and turning on a second set of spatial sampling filters for the non-subpixel rendered areas in response to detecting the non-subpixel rendered areas.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a divisional of, and claims the benefit of thefiling date of, U.S. patent application Ser. No. 10/051,612, entitled“Conversion of a Sub-Pixel Format Data to Another Sub-Pixel DataFormat”, filed on Jan. 16, 2002, and subsequently issued as U.S. Pat.No. 7,123,277 and which is hereby incorporated by reference herein. U.S.patent application Ser. No. 10/051,612 incorporated by reference andclaimed the benefit of the filing dates of U.S. U.S. Provisional PatentApplication Ser. No. 60/290,086, entitled “Conversion of RGB PixelFormat Data to Pentile Matrix Sub-Pixel Data Format”, filed on May 9,2001, U.S. Provisional Patent Application Ser. No. 60/290,087, entitled“Calculating Filter Kernel Values for Different Scaled Modes”, filed onMay 9, 2001, U.S. Provisional Patent Application Ser. No. 60/290,143,entitled “Scaling Sub-Pixel Rendering on Pentile Matrix”, filed on May9, 2001, and U.S. Provisional Patent Application Ser. No. 60/313,054,entitled “RGB Stripe Sub-Pixel Rendering Detection”, filed on Aug. 16,2001, all of whose disclouseres are also incorporated by referenceherein in their entirety.

BACKGROUND

The present application relates to the conversion of graphics dataformats from one form to another, and specifically to the conversion of(red-green-blue) RGB graphics to improved color pixel arrangements usedin displays.

The present state of the art of color single plane imaging matrix, forflat panel displays, use the RGB color triad or a single color in avertical stripe as shown in prior art FIG. 1. The system takes advantageof the Von Bezold color blending effect (explained further herein) byseparating the three colors and placing equal spatial frequency weighton each color. However, these panels are a poor match to human vision.

Graphic rendering techniques have been developed to improve the imagequality of prior art panels. Benzschawel, et al. in U.S. Pat. No.5,341,153 teach how to reduce an image of a larger size down to asmaller panel. In so doing, Benzschawel, et al. teach how to improve theimage quality using a technique now known in the art as “sub-pixelrendering”. More recently Hill, et al. in U.S. Pat. No. 6,188,385 teachhow to improve text quality by reducing a virtual image of text, onecharacter at a time, using the very same sub-pixel rendering technique.

The above prior art pay inadequate attention to how human visionoperates. The prior art's reconstruction of the image by the displaydevice is poorly matched to human vision.

The dominant model used in sampling, or generating, and then storing theimage for these displays is the RGB pixel (or three-color pixelelement), in which the red, green, and blue values are on an orthogonalequal spatial resolution grid and are co-incident. One of theconsequences of using this image format is that it is a poor match bothto the real image reconstruction panel, with its spaced apart,non-coincident, color emitters, and to human vision. This effectivelyresults in redundant, or wasted, information in the image.

Martinez-Uriegas, et al. in U.S. Pat. No. 5,398,066 and Peters, et al.in U.S. Pat. No. 5,541,653 teach a technique to convert and store imagesfrom RGB pixel format to a format that is very much like that taught byBayer in U.S. Pat. No. 3,971,065 for a color filter array for imagingdevices for cameras. The advantage of the Martinez-Uriegas, et al.format is that it both captures and stores the individual colorcomponent data with similar spatial sampling frequencies as humanvision. However, a first disadvantage is that the Martinez-Uriegas, etal. format is not a good match for practical color display panels. Forthis reason, Martinez-Uriegas, et al. also teach how to convert theimage back into RGB pixel format. Another disadvantage of theMartinez-Uriegas, et al. format is that one of the color components, inthis case the red, is not regularly sampled. There are missing samplesin the array, reducing the accuracy of the reconstruction of the imagewhen displayed.

Full color perception is produced in the eye by three-color receptornerve cell types called cones. The three types are sensitive todifferent wave lengths of light: long, medium, and short (“red”,“green”, and “blue”, respectively). The relative density of the threewavelengths differs significantly from one another. There are slightlymore red receptors than green receptors. There are very few bluereceptors compared to red or green receptors. In addition to the colorreceptors, there are relative wavelength insensitive receptors calledrods that contribute to monochrome night vision.

The human vision system processes the information detected by the eye inseveral perceptual channels: luminance, chromanance, and motion. Motionis only important for flicker threshold to the imaging system designer.The luminance channel takes the input from only the red and greenreceptors. It is “color blind”. It processes the information in such amanner that the contrast of edges is enhanced. The chromanance channeldoes not have edge contrast enhancement. Since the luminance channeluses and enhances every red and green receptor, the resolution of theluminance channel is several times higher than the chromanance channel.The blue receptor contribution to luminance perception is negligible.Thus, the error introduced by lowering the blue resolution by one octavewill be barely noticeable by the most perceptive viewer, if at all, asexperiments at Xerox and NASA, Ames Research Center (R. Martin, J.Gille, J. Larimer, Detectability of Reduced Blue Pixel Count inProjection Displays, SID Digest 1993) have demonstrated.

Color perception is influenced by a process called “assimilation” or theVon Bezold color blending effect. This is what allows separate colorpixels (or sub-pixels or emitters) of a display to be perceived as themixed color. This blending effect happens over a given angular distancein the field of view. Because of the relatively scarce blue receptors,this blending happens over a greater angle for blue than for red orgreen. This distance is approximately 0.25°. for blue, while for red orgreen it is approximately 0.12°. At a viewing distance of twelve inches,0.25°. subtends 50 mils (1,270μ) on a display. Thus, if the bluesub-pixel pitch is less than half (625μ.) of this blending pitch, thecolors will blend without loss of picture quality.

Sub-pixel rendering, in its most simplistic implementation, operates byusing the sub-pixels as approximately equal brightness pixels perceivedby the luminance channel. This allows the sub-pixels to serve as sampledimage reconstruction points as opposed to using the combined sub-pixelsas part of a ‘true’ pixel. By using sub-pixel rendering, the spatialsampling is increased, reducing the phase error.

If the color of the image were to be ignored, then each sub-pixel mayserve as a though it were a monochrome pixel, each equal. However, ascolor is nearly always important (and why else would one use a colordisplay?), then color balance of a given image is important at eachlocation. Thus, the sub-pixel rendering algorithm must maintain colorbalance by ensuring that high spatial frequency information in theluminance component of the image to be rendered does not alias with thecolor sub-pixels to introduce color errors. The approaches taken byBenzschawel, et al. in U.S. Pat. No. 5,341,153, and Hill, et al. in U.S.Pat. No. 6,188,385, are similar to a common anti-aliasing technique thatapplies displaced decimation filters to each separate color component ofa higher resolution virtual image. This ensures that the luminanceinformation does not alias within each color channel.

If the arrangement of the sub-pixels were optimal for sub-pixelrendering, sub-pixel rendering would provide an increase in both spatialaddressability to lower phase error and in Modulation Transfer Function(MTF) high spatial frequency resolution in both axes.

Examining the conventional RGB stripe display in FIG. 1, sub-pixelrendering will only be applicable in the horizontal axis. The bluesub-pixel is not perceived by the human luminance channel, and istherefore, not effective in sub-pixel rendering. Since only the red andgreen pixels are useful in sub-pixel rendering, the effective increasein addressability would be two-fold, in the horizontal axis. Verticalblack and white lines must have the two dominant sub-pixels (i.e., redand green per each black or white line) in each row. This is the samenumber as is used in non-sub-pixel rendered images. The MTF, which isthe ability to simultaneously display a given number of lines andspaces, is not enhanced by sub-pixel rendering. Thus, the conventionalRGB stripe sub-pixel arrangement, as shown in FIG. 1, is not optimal forsub-pixel rendering.

The prior art arrangements of three-color pixel elements are shown to beboth a poor match to human vision and to the generalized technique ofsub-pixel rendering. Likewise, the prior art image formats andconversion methods are a poor match to both human vision and practicablecolor emitter arrangements.

SUMMARY

The drawbacks and disadvantages of the prior art are overcome by theconversion of RGB pixel format data to PenTile™ matrix sub-pixel dataformat.

A method of converting a source pixel data of a first format for adisplay of a second format having a plurality of three-color pixelelements is disclosed. The method comprises determining implied sampleareas for each data point of each color in the source pixel data of thefirst format. The resample areas for each emitter of each color in thedisplay is also determined. A set of fractions for each resample area isformed. The denominators are a function of the resample area and thenumerators are the function of an area of each of the implied sampleareas that at least partially overlaps the resample areas. The datavalues for each implied sample area is multiplied by its respectivefraction and all products are added together to obtain luminance valuesfor each resample area.

A method of determining implied sample areas for each data point of eachcolor in a source pixel data of a first format for a display of a secondformat having a plurality of three-color pixel elements is alsodisclosed. The method comprises determining a geometric center of eachemitter of each of the three-color pixel element of the first format todefine sampling points. Then defining each of the implied sample area bylines that are formed equidistant between the geometric center of theemitter of one the three-color pixel element and the geometric center ofanother same color the emitter of a neighboring three-color pixelelement and forming a grid of the lines.

A method of limiting filter kernel divisors in a filter kernel to avalue designed to simplify hardware implementations is also disclosed.The method comprises calculating areas for filter coefficients usingfloating point arithmetic and then dividing each filter coefficient by atotal area of a rendering area to receive a first product. Thenmultiplying the first product by a divisor to produce a filter sum,completing a binary search to find a round off point for the filter sum,and converting the filter sum to integers.

BRIEF DESCRIPTION OF THE DRAWINGS

Referring now to the figures, wherein like elements are numbered alike:

FIG. 1 illustrates a prior art RGB stripe arrangement of three-colorpixel elements in an array, a single plane, for a display device;

FIG. 2 illustrates the effective sub-pixel rendering sampling points forthe prior art RGB stripe arrangement of FIG. 1;

FIGS. 3, 4, and 5 illustrate the effective sub-pixel rendering samplingarea for each color plane of the sampling points for the prior art RGBstripe arrangement of FIG. 1.

FIG. 6 illustrates an arrangement of three-color pixel elements in anarray, in a single plane, for a display device;

FIG. 7 illustrates the effective sub-pixel rendering sampling points forthe arrangements of FIGS. 6 and 27;

FIGS. 8 and 9 illustrate alternative effective sub-pixel renderingsampling areas for the blue color plane sampling points for thearrangements of FIGS. 6 and 27;

FIG. 10 illustrates another arrangement of three-color pixel elements inan array, in a single plane, for a display device;

FIG. 11 illustrates the effective sub-pixel rendering sampling pointsfor the arrangement of FIG. 10;

FIG. 12 illustrates the effective sub-pixel rendering sampling areas forthe blue color plane sampling points for the arrangement of FIG. 10;

FIGS. 13 and 14 illustrate the effective sub-pixel rendering samplingareas for the red and green color planes for the arrangements of bothFIGS. 6 and 10;

FIG. 15 illustrates an array of sample points and their effective sampleareas for a prior art pixel data format, in which the red, green, andblue values are on an equal spatial resolution grid and co-incident;

FIG. 16 illustrates the array of sample points of prior art FIG. 15overlaid on the sub-pixel rendered sample points of FIG. 11, in whichthe sample points of FIG. 15 are on the same spatial resolution grid andco-incident with the red and green “checker board” array of FIG. 11;

FIG. 17 illustrates the array of sample points and their effectivesample areas of prior art FIG. 15 overlaid on the blue color planesampling areas of FIG. 12, in which the sample points of prior art FIG.15 are on the same spatial resolution grid and co-incident with the redand green “checker board” array of FIG. 11;

FIG. 18 illustrates the array of sample points and their effectivesample areas of prior art FIG. 15 overlaid on the red color planesampling areas of FIG. 13, in which the sample points of prior art FIG.15 are on the same spatial resolution grid and co-incident with the redand green “checker board” array of FIG. 11;

FIGS. 19 and 20 illustrate the array of sample points and theireffective sample areas of prior art FIG. 15 overlaid on the blue colorplane sampling areas of FIGS. 8 and 9, in which the sample points ofprior art FIG. 15 are on the same spatial resolution grid andco-incident with the red and green “checker board” array of FIG. 7;

FIG. 21 illustrates an array of sample points and their effective sampleareas for a prior art pixel data format in which the red, green, andblue values are on an equal spatial resolution grid and co-incident;

FIG. 22 illustrates the array of sample points and their effectivesample areas of prior art FIG. 21 overlaid on the red color planesampling areas of FIG. 13, in which the sample points of FIG. 21 are noton the same spatial resolution grid and co-incident with the red andgreen “checker board” array of FIG. 11;

FIG. 23 illustrates the array of sample points and their effectivesample areas of prior art FIG. 21 overlaid on the blue color planesampling areas of FIG. 12, in which the sample points of prior art FIG.21 are not on the same spatial resolution grid nor co-incident with thered and green “checker board” array of FIG. 11;

FIG. 24 illustrates the array of sample points and their effectivesample areas of prior art FIG. 21 overlaid on the blue color planesampling areas of FIG. 8, in which the sample points of prior art FIG.21 are not on the same spatial resolution grid nor co-incident with thered and green “checker board” array of FIG. 7;

FIG. 25 illustrates the effective sample area of the red color plane ofFIG. 3 overlaid on the red color plane sampling areas of FIG. 13;

FIG. 26 illustrates the effective sample areas of the blue color planeof FIG. 5 overlaid on the blue color plane sampling areas of FIG. 8

FIG. 27 illustrates another arrangement of three-color pixel elements inan array, in three panels, for a display device;

FIGS. 28, 29, and 30 illustrate the arrangements of the blue, green, andred emitters on each separate panel for the device of FIG. 27;

FIG. 31 illustrates the output sample arrangement 200 of FIG. 11overlaid on top of the input sample arrangement 70 of FIG. 15 in thespecial case when the scaling ratio is one input pixel for each two, ared and a green, output sub pixels across;

FIG. 32 illustrates a single repeat cell 202 of converting a 640×480 VGAformat image to a PenTile matrix with 800×600 total red and green subpixels;

FIG. 33 illustrates the symmetry in the coefficients of a three-colorpixel element in a case where the repeat cell size is odd;

FIG. 34 illustrates an example of a case where the repeat cell size iseven;

FIG. 35 illustrates sub-pixel 218 from FIG. 33 bounded by a renderingarea 246 that overlaps six of the surrounding input pixel sample areas248;

FIG. 36 illustrates sub-pixel 232 from FIG. 33 with its rendering area250 overlapping five sample areas 252;

FIG. 37 illustrates sub-pixel 234 from FIG. 33 with its rendering area254 overlapping sample areas 256;

FIG. 38 illustrates sub-pixel 228 from FIG. 33 with its rendering area258 overlapping sample areas 260;

FIG. 39 illustrates sub-pixel 236 from FIG. 33 with its rendering area262 overlapping sample areas 264;

FIG. 40 illustrates the square sampling areas used for generating bluefilter kernels; and

FIG. 41 illustrates the hexagonal sampling areas 123 of FIG. 8 inrelationship to the square sampling areas 276.

DETAILED DESCRIPTION

Those of ordinary skill in the art will realize that the followingdescription of the present invention is illustrative only and not in anyway limiting. Other embodiments of the invention will readily suggestthemselves to such skilled persons.

A real world image is captured and stored in a memory device. The imagethat is stored was created with some known data arrangement. The storedimage can be rendered onto a display device using an array that providesan improved resolution of color displays. The array is comprised of aplurality of three-color pixel elements having at least a blue emitter(or sub-pixel), a red emitter, and a green emitter, which whenilluminated can blend to create all other colors to the human eye.

To determine the values for each emitter, first one must createtransform equations that take the form of filter kernels. The filterkernels are generated by determining the relative area overlaps of boththe original data set sample areas and target display sample areas. Theratio of overlap determines the coefficient values to be used in thefilter kernel array.

To render the stored image onto the display device, the reconstructionpoints are determined in each three-color pixel element. The center ofeach reconstruction point will also be the source of sample points usedto reconstruct the stored image. Similarly, the sample points of theimage data set is determined. Each reconstruction point is located atthe center of the emitters (e.g., in the center of a red emitter). Inplacing the reconstruction points in the center of the emitter, a gridof boundary lines is formed equidistant from the centers of thereconstruction points, creating sample areas (in which the sample pointsare at the center). The grid that is formed creates a tiling pattern.The shapes that can be utilized in the tiling pattern can include, butis not limited to, squares, rectangles, triangles, hexagons, octagons,diamonds, staggered squares, staggered rectangles, staggered triangles,staggered diamonds, Penrose tiles, rhombuses, distorted rhombuses, andthe like, and combinations comprising at least one of the foregoingshapes.

The sample points and sample areas for both the image data and thetarget display having been determined, the two are overlaid. The overlaycreates sub-areas wherein the output sample areas overlap several inputsample areas. The area ratios of input to output is determined by eitherinspection or calculation and stored as coefficients in filter kernels,the value of which is used to weight the input value to output value todetermine the proper value for each emitter.

When sufficiently high scaling ratio is used, the subpixel arrangementand rendering method disclosed herein provides better image quality,measured in information addressability and reconstructed imagemodulation transfer function (MTF), than prior art displays.

FIG. 1 illustrates a prior art RGB stripe arrangement of three-colorpixel elements in an array, a single plane, for a display device andprior art FIG. 2 illustrates the effective sub-pixel rendering samplingpoints for the prior art RGB stripe arrangement of FIG. 1. Prior artFIGS. 3, 4, and 5 illustrate the effective sub-pixel rendering samplingarea for each color plane of the sampling points for the prior art RGBstripe arrangement of FIG. 1. FIGS. 1-5 will be discussed furtherherein.

FIG. 6 illustrates an arrangement 20 of several three-color pixelelements according to one embodiment. The three-color pixel element 21is square-shaped and disposed at the origin of an X, Y coordinate systemand comprises a blue emitter 22, two red emitters 24, and two greenemitters 26. The blue emitter 22 is disposed at the center, verticallyalong the X axis, of the coordinate system extending into the first,second, third, and fourth quadrants. The red emitters 24 are disposed inthe second and fourth quadrants, not occupied by the blue emitter. Thegreen emitters 26 are disposed in the first and third quadrants, notoccupied by the blue emitter. The blue emitter 22 is rectangular-shaped,having sides aligned along the X and Y axes of the coordinate system,and the opposing pairs of red 24 and green 26 emitters are generallysquare-shaped.

The array is repeated across a panel to complete a device with a desiredmatrix resolution. The repeating three-color pixel elements form a“checker board” of alternating red 24 and green 26 emitters with blueemitters 22 distributed evenly across the device, but at half theresolution of the red 24 and green 26 emitters. Every other column ofblue emitters is staggered, or shifted by half of its length, asrepresented by emitter 28. To accommodate this and because of edgeeffects, some of the blue emitters are half-sized blue emitters 28 atthe edges.

FIG. 7 illustrates an arrangement 29 of the effective sub-pixelrendering sampling points for the arrangements of FIGS. 6 and 27, whileFIGS. 8 and 9 illustrate arrangements 30, 31 of alternative effectivesub-pixel rendering sampling areas 123, 124 for the blue color planesampling points 23 for the arrangements of FIGS. 6 and 27. FIGS. 7, 8,and 9 will be discussed further herein.

FIG. 10 illustrates an alternative illustrative embodiment of anarrangement 38 of three-color pixel elements 39. The three-color pixelelement 39 consists of a blue emitter 32, two red emitters 34, and twogreen emitters 36 in a square. The three-color pixel element 39 issquare shaped and is centered at the origin of an X, Y coordinatesystem. The blue emitter 32 is centered at the origin of the square andextends into the first, second, third, and fourth quadrants of the X, Ycoordinate system. A pair of red emitters 34 are disposed in opposingquadrants (i.e., the second and the fourth quadrants), and a pair ofgreen emitters 36 are disposed in opposing quadrants (i.e., the firstand the third quadrants), occupying the portions of the quadrants notoccupied by the blue emitter 32. As shown in FIG. 10, the blue emitter32 is diamond shaped, having corners aligned at the X and Y axes of thecoordinate system, and the opposing pairs of red 34 and green 36emitters are generally square shaped, having truncated inwardly-facingcorners forming edges parallel to the sides of the blue emitter 32.

The array is repeated across a panel to complete a device with a desiredmatrix resolution. The repeating three-color pixels form a “checkerboard” of alternating red 34 and green 36 emitters with blue emitters 32distributed evenly across the device, but at half the resolution of thered 34 and green 36 emitters. Red emitters 34 a and 34 b will bediscussed further herein.

One advantage of the three-color pixel element array is an improvedresolution of color displays. This occurs since only the red and greenemitters contribute significantly to the perception of high resolutionin the luminance channel. Thus, reducing the number of blue emitters andreplacing some with red and green emitters improves resolution by moreclosely matching to human vision.

Dividing the red and green emitters in half in the vertical axis toincrease spatial addressability is an improvement over the conventionalvertical single color stripe of the prior art. An alternating “checkerboard” of red and green emitters allows high spatial frequencyresolution, to increase in both the horizontal and the vertical axes.

In order to reconstruct the image of the first data format onto thedisplay of the second data format, sample areas need to be defined byisolating reconstruction points in the geometric center of each emitterand creating a sampling grid. FIG. 11 illustrates an arrangement 40 ofthe effective reconstruction points for the arrangement 38 ofthree-color pixel elements 39 of FIG. 10. The reconstruction points 33,35, and 37 of FIG. 11 are centered over the geometric locations of theemitters 32, 34, and 36, respectively in the three-color pixel element39 of FIG. 10. The red reconstruction points 35 and the greenreconstruction points 37 form a red and green “checker board” arrayacross the display. The blue reconstruction points 33 are distributedevenly across the device, but at half the resolution of the red 35 andgreen 37 reconstruction points. For sub-pixel rendering, these colorreconstruction points are treated as sampling points and are used toconstruct the effective sampling area for each color plane, which aretreated separately. FIG. 12 illustrates the effective blue samplingpoints 46 (corresponding to blue reconstruction point 33 of FIG. 11) andsampling areas 44 for the blue color plane 42 for the reconstructionarray of FIG. 11. For a square grid of reconstruction points, theminimum boundary perimeter is a square grid.

FIG. 13 illustrates the effective red sampling points 51 that correspondto the red reconstruction points 35 of FIG. 11 and to the redreconstruction points 25 of FIG. 7, and the effective sampling areas 50,52, 53, and 54 for the red color plane 48. The sampling points 51 form asquare grid array at 45° to the display boundary. Thus, within thecentral array of the sampling grid, the sampling areas form a squaregrid. Because of ‘edge effects’ where the square grid would overlap theboundary of the display, the shapes are adjusted to keep the same areaand minimize the boundary perimeter of each sample (e.g., 54).Inspection of the sample areas will reveal that sample areas 50 have thesame area as sample areas 52, however, sample areas 54 has slightlygreater area, while sample areas 53 in the corners have slightly less.This does introduce an error, in that the varying data within the sampleareas 53 will be over represented while varying data in sample areas 54will be under represented. However, in a display of hundreds ofthousands to millions of emitters, the error will be minimal and lost inthe corners of the image.

FIG. 14 illustrates the effective green sampling points 57 thatcorrespond to the green reconstruction points 37 of FIG. 11 and to thegreen reconstruction points 27 of FIG. 7, and the effective samplingareas 55, 56, 58, and 59 for the green color plane 60. Inspection ofFIG. 14 will reveal it is essential similar to FIG. 13, it has the samesample area relationships, but is rotated by 180°.

These arrangements of emitters and their resulting sample points andareas would best be used by graphics software directly to generate highquality images, converting graphics primitives or vectors to offsetcolor sample planes, combining prior art sampling techniques with thesampling points and areas. Complete graphics display systems, such asportable electronics, laptop and desktop computers, and television/videosystems, would benefit from using flat panel displays and these dataformats. The types of displays utilized can include, but is not limitedto, liquid crystal displays, subtractive displays, plasma paneldisplays, electro-luminecence (EL) displays, electrophoretic displays,field emitter displays, discrete light emitting diode displays, organiclight emitting diodes (OLEDs) displays, projectors, cathode ray tube(CRT) displays, and the like, and combinations comprising at least oneof the foregoing displays. However, much of the installed base ofgraphics and graphics software uses a legacy data sample formatoriginally based on the use of CRTs as the reconstruction display.

FIG. 15 illustrates an array 70 of sample points 74 and their effectivesample areas 72 for a prior art pixel data format in which the red,green, and blue values are on an equal spatial resolution grid andco-incident. In prior art display systems, this form of data wasreconstructed on a flat panel display by simply using the data from eachcolor plane on a prior art RGB stripe panel of the type shown in FIG. 1.In FIG. 1, the resolution of each color sub-pixel was the same as thesample points, treating three sub-pixels in a row as though theyconstituted a single combined and intermingled multi-color pixel whileignoring the actual reconstruction point positions of each colorsub-pixel. In the art, this is often referred to as the “Native Mode” ofthe display. This wastes the positional information of the sub-pixels,especially the red and green.

In contrast, the incoming RGB data of the present application is treatedas three planes over lying each other. To covert the data from the RGBformat, each plane is treated separately. Displaying information fromthe original prior art format on the more efficient sub-pixelarrangements of the present application requires a conversion of thedata format via resampling. The data is resampled in such a fashion thatthe output of each sample point is a weighting function of the inputdata. Depending on the spatial frequency of the respective data samples,the weighting function may be the same, or different, at each outputsample point, as will be described below.

FIG. 16 illustrates the arrangement 76 of sample points 74 of FIG. 15overlaid on the effective reconstruction points 33, 35, and 37 of FIG.11, in which the sample points 74 of FIG. 15 are on the same spatialresolution grid and co-incident with the red (red reconstruction points35) and green (green reconstruction points 37) “checker board” array ofFIG. 11.

FIG. 17 illustrates the arrangement 78 of sample points 74 and theireffective sample areas 72 of FIG. 15 overlaid on the blue color planesampling points 46 of FIG. 12, in which the sample points 74 of FIG. 15are on the same spatial resolution grid and co-incident with the red(red reconstruction points 35) and green (green reconstruction points37) “checker board” array of FIG. 11. FIG. 17 will be discussed furtherherein.

FIG. 18 illustrates the array 80 of sample points 74 and their effectivesample areas 72 of FIG. 15 overlaid on the red color planereconstruction points 35 and the red output sample areas 50, 52, 53, and54 of FIG. 13, in which the sample points 74 of FIG. 15 are on the samespatial resolution grid and co-incident with the red (red reconstructionpoints 35) and green (green reconstruction points 37) “checker board”array of FIG. 11. Each of the inner array of square red sample areas 52completely covers the coincident original sample point 74 and its samplearea 72 as well as extending to cover one quarter each of thesurrounding sample areas 84 that lie inside the sample area 52. Todetermine the algorithm, the fraction of coverage, or overlap, of theoutput sample area 50, 52, 53, or 54 over the input sample area 72 isrecorded and then multiplied by the value of that corresponding samplepoint 74 and applied to the red reconstruction point 35. In FIG. 18, thearea of red output sample area 52 filled by the central, or coincident,input sample area 72 is half of square sample area 52. Thus, the valueof the corresponding sample point 74 is multiplied by one half (or 0.5).By inspection, the area of square sample area 52 filled by each of thesurrounding, non-coincident, input areas 84 is one eighth (or 0.125)each. Thus, the value of the corresponding four input sample points 74is multiplied by one eighth (or 0.125). These values are then added tothe previous value (e.g., that was multiplied by 0.5) to find the finaloutput value of a given red reconstruction point 35 in sample area 52.

For the red reconstruction points 35 on the edges and their five sidedsample areas 50, the coincident input sample area 72 is completelycovered as in the case described above, but only three surrounding inputsample areas 85, 86, and 92 are overlapped. One of the overlapped inputsample areas 85 represents one eighth of the output sample area 50. Theneighboring input sample areas 86 and 92 along the edge represent threesixteenths ( 3/16=0.1875) of the output area each. As before, theweighted values of the input values 74 from the overlapped sample areas72 are added to give the value for red reconstruction point 35 in samplearea 50.

The corners and “near” corners are treated the same. Since the areas ofthe image that the corner sample areas 53 and “near” corner sample areas54 cover are different than the central sample areas 52 and edge sampleareas 50, the weighting of the relevant input sample areas will bedifferent than those previously described. For the smaller corner outputsample areas 53, the coincident input sample area 72 covers foursevenths (or about 0.5714) of output sample area 53. The neighboringinput sample areas 96 cover three fourteenths (or about 0.2143) of theoutput sample area 53. For the “near” corner sample areas 54, thecoincident input sample area 72 covers eight seventeenths (or about0.4706) of the output sample area 54. The inward neighboring sample area98 covers two seventeenths (or about 0.1176) of the output sample area54. The edge wise neighboring input sample area 95 covers threeseventeenths (or about 0.1765) of the output sample area 54. The cornerinput sample area 97 covers four seventeenths (or about 0.2353) of theoutput sample area 54. As before, the weighted values of the inputvalues 74 from the overlapped sample areas 72 are added to give thevalue for red reconstruction point 35.

The calculation for the resampling of the green color plane proceeds ina similar manner, but the output sample array is rotated by 180°.

To restate, the calculations for the red sample point 35 and greensample point 37 values, V_(out), are as follows

Central Areas:V _(out)(C _(x) R _(y))=0.5_(—) V _(in)(C _(x) R _(y))+0.125_(—) V_(in)(C _(x−1) R _(y))+0.125_(—) V _(in)(C _(x) R _(y+1))+0.125_(—) V_(in)(C _(x+1) R _(y))+0.125_(—) V _(in)(C _(x) R _(y−1))

Lower Edge:V _(out)(C _(x) R _(y))=0.5_(—) V _(in)(C _(x) R _(y))+0.1875_(—) V_(in)(C _(x) R _(y+1))+0.125_(—) V _(in)(C _(x+1) R _(y))

Upper Edge:V _(out)(C _(x) R ₁)=0.5_(—) V _(in)(C _(x) R ₁)+0.1875_(—) V _(in)(C_(x−1) R1)+0.125_(—) V _(in)(C _(x) R ₂)+0.1875_(—) V _(in)(C _(x+1) R1)

Right Edge:V _(out)(C _(x) R _(y))=0.5_(—) V _(in)(C _(x) R _(y))+0.125_(—) V_(in)(C _(x−1) R _(y))+0.1875_(—) V _(in)(C _(x) R _(y+1))+0.1875_(—) V_(in)(C _(x) R _(y))

Left EdgeV _(out)(C ₁ R _(y))=0.5_(—) V _(in)(C ₁ R _(y))+0.1875_(—) V _(in)(C ₁R _(y+1))+0.125_(—) V _(in)(C ₂ R _(y))+0.1875_(—) V _(in)(C ₁ R _(y−1))

Upper Right Hand Corner:V _(out)(C _(x) R _(y))=0.5714_(—) V _(in)(C _(x) R _(y))+0.2143_(—) V_(in)(C _(x−1) R _(y))+0.2143_(—) V _(in)(C _(x) R _(y+1))

Upper Left Hand Corner:V _(out)(C ₁ R ₁)=0.5714_(—) V _(in)(C ₁ R ₁)+0.2143_(—) V _(in)(C ₁ R₂)+0.2143_(—) V _(in)(C ₂ R ₁)

Lower Left Hand Corner:V _(out)(C _(x) R _(y))=0.5714_(—) V _(in)(C _(x) R _(y))+0.2143_(—) V_(in)(C _(x−1) R _(y))+0.2143_(—) V _(in)(C _(x) R _(y−1))

Lower Right Hand Corner:V _(out)(C _(x) R _(y))=0.5714_(—) V _(in)(C _(x) R _(y))+0.2143_(—) V_(in)(C _(x−1) R _(y))+0.2143—V _(in)(C _(x) R _(y−1))

Upper Edge, Left Hand Near Corner:V _(out)(C ₂ R ₁)=0.4706_(—) V _(in)(C ₂ R ₁)+0.2353_(—) V _(in)(C ₁ R₁)+0.1176_(—) V _(in)(C ₂ R ₂)+0.1765_(—) V _(in)(C ₃ R ₁)

Left Edge, Upper Near Corner:V _(out)(C ₁ R ₂)=0.4706_(—) V _(in)(C ₁ R ₂)+0.1765_(—) V _(in)(C ₁ R₃)+0.1176_(—) V _(in)(C ₂ R ₂)+0.2353_(—) V _(in)(C ₁ R ₁)

Left Edge Lower Near Corner:V _(out)(C ₁ R _(y))=0.4706_(—) V _(in)(C ₁ R _(y))+0.2353_(—) V _(in)(C₁ R _(y+1))+0.1176_(—) V _(in)(C ₂ R _(y))+0.1765_(—) V _(in)(C ₁ R_(y−1))

Lower Edge, Left Hand Near Corner:V _(out)(C ₂ R _(y))=0.4706_(—) V _(in)(C ₂ R _(y))+0.2353_(—) V _(in)(C₁ R _(y))+0.1176_(—) V _(in)(C ₂ R _(y−1))+0.125_(—) V _(in)(C _(x) R_(y−1))

Lower Edge, Right Hand Near Corner:V _(out)(C _(x) R _(y))=0.4706_(—) V _(in)(C _(x) R _(y))+0.1765_(—) V_(in)(C _(x−1) R _(y))+0.2353_(—) V _(in)(C _(x+1) R _(y))+0.1176_(—) V_(in)(C _(x) R _(y−1))

Right Edge, Lower Near Corner:V _(out)(C _(x) R _(y))=0.4706_(—) V _(in)(C _(x) R _(y))+0.1176_(—) V_(in)(C _(x−1) R _(y))+0.2353_(—) V _(in)(C _(x) R _(y+1))+0.1765_(—) V_(in)(C _(x) R _(y−1))

Right Edge, Upper Near Corner:V _(out)(C _(x) R ₂)=0.4706_(—) V _(in)(C _(x) R ₂)+0.1176_(—) V _(in)(C_(x−1) R ₂)+0.1765_(—) V _(in)(C _(x) R ₃)+0.2353_(—) V _(in)(C _(x) R₁)

Upper Edge, Right Hand Near Corner:V _(out)(C _(x) R ₁)=0.4706_(—) V _(in)(C _(x) R ₁)+0.1765_(—) V _(in)(C_(x−1) R ₁)+_(—) V _(in)(C _(x−1) R ₁)+0.1176_(—) V _(in)(C _(x) R₂)+0.2353_(—) V _(in)(C _(x+1) R ₁)where V_(in) are the chrominance values for only the color of thesub-pixel at C_(x)R_(y) (C_(x) represents the x^(th) column of red 34and green 36 sub-pixels and R_(y) represents the y^(th) row of red 34and green 36 sub-pixels, thus C_(x)R_(y) represents the red 34 or green36 sub-pixel emitter at the x^(th) column and y^(th) row of the displaypanel, starting with the upper left-hand corner, as is conventionallydone). It is important to note that the total of the coefficient weightsin each equation add up to a value of one. Although there are seventeenequations to calculate the full image conversion, because of thesymmetry there are only four sets of coefficients. This reduces thecomplexity when implemented.

As stated earlier, FIG. 17 illustrates the arrangement 78 of samplepoints 74 and their effective sample areas 72 of FIG. 15 overlaid on theblue color plane sampling points 46 of FIG. 12, in which the samplepoints 74 of FIG. 15 are on the same spatial resolution grid andco-incident with the red (red reconstruction points 35) and green (greenreconstruction points 37) “checker board” array of FIG. 11. The bluesampling points 46 of FIG. 12 allow each blue sample area 44 (shown inbold) to be determined by inspection. In this case, the blue sample area44 is now a blue resample area which is simply the arithmetic mean ofthe surrounding blue values of the original data sample points 74 thatis computed as the value for the blue sampling point 46 of the resampledimage.

The blue output value, V_(out), of a blue sampling point 46 iscalculated as follows:V _(out)(C _(x+) _(—) R _(y+))=0.25_(—) V _(in)(C _(x) R _(y))+0.25_(—)V _(in)(C _(x) R _(y+1))+0.25_(—) V _(in)(C _(x+1) R _(y))+0.25_(—) V_(in)(C _(x+1) R _(y+1))where Vin are the blue chromanance values of the surrounding inputsample points 74; C_(x) represents the x^(th) column of sample points74; and R_(y) represents the y^(th) row of sample points 74, startingwith the upper left-hand corner, as is conventionally done.

For the blue sub-pixel calculation, x and y numbers must be odd, asthere is only one blue sub-pixel per pairs of red and green sub-pixels.Again, the total of the coefficient weights is equal to a value of one.

The weighting of the coefficients of the central area equation for thered sample point 35, which affects most of the image created, andapplying to the central resample areas 52 is the process of binary shiftdivision, where 0.5 is a one bit shift to the “right”, 0.25 is a two bitshift to the “right”, and 0.125 is a three bit shift to the “right”.Thus, the algorithm is extremely simple and fast, involving simple shiftdivision and addition. For greatest accuracy and speed, the addition ofthe surrounding pixels should be completed first, followed by a singlethree bit shift to the right, and then the single bit shifted centralvalue is added. However, the latter equations for the red and greensample areas at the edges and the corners involve more complexmultiplications. On a small display (e.g., a display having few totalpixels), a more complex equation may be needed to ensure good imagequality display. For large images or displays, where a small error atthe edges and corner may matter very little, a simplification may bemade. For the simplification, the first equation for the red and greenplanes is applied at the edges and corners with the “missing” input datasample points over the edge of the image, such that input sample points74 are set to equal the coincident input sample point 74. Alternatively,the “missing” values may be set to black. This algorithm may beimplemented with ease in software, firmware, or hardware.

It is important that the chromanance values be linearly additive,meaning that the sub-pixel rendering must be completed before gammacorrection. The outputs of the above algorithm may feed into the gammacorrection tables. If gamma correction is performed before sub-pixelrendering, unexpected chromanance errors are likely to occur.

FIGS. 19 and 20 illustrate two alternative arrangements 100, 102 ofsample points 74 and their effective sample areas 72 of FIG. 15 overlaidon the blue color plane sampling areas 23 of FIGS. 8 and 9, in which thesample points 74 of FIG. 15 are on the same spatial resolution grid andco-incident with the red and green “checker board” array of FIG. 7. FIG.8 illustrates the effective sub-pixel rendering sampling areas 123 thathave the minimum boundary perimeters for the blue color plane samplingpoints 23 shown in FIG. 7 for the arrangement of emitters in FIG. 6.

The method for calculating the coefficients proceeds as described above.The proportional overlap of output sample areas 123 in that overlap eachinput sample area 72 of FIG. 19 are calculated and used as coefficientsin a transform equation or filter kernel. These coefficients aremultiplied by the sample values 74 in the following transform equation:

$\begin{matrix}{{V_{out}( {C_{x^{+}}{\_ R}_{y^{+}}\_} )} = {{0.015625{\_ V}_{in}( {C_{x - 1}R_{y}} )} +}} \\{{0.234375{\_ V}_{in}( {C_{x}R_{y}} )} +} \\{{0.234375{\_ V}_{in}( {C_{x + 1}R_{y}} )} +} \\{{0.015625{\_ V}_{in}( {C_{x + 2}R_{y}} )} +} \\{{0.234375{\_ V}_{in}( {C_{x}R_{y + 1}} )} +} \\{{0.234375{\_ V}_{in}( {C_{x + 1}R_{y + 1}} )} +} \\{0.015625{\_ V}_{in}{( {C_{X + 2}R_{y + 1}} ).}}\end{matrix}$

A practitioner skilled in the art can find ways to perform thesecalculations rapidly. For example, the coefficient 0.015625 isequivalent to a 6 bit shift to the right. In the case where samplepoints 74 of FIG. 15 are on the same spatial resolution grid andco-incident with the red (red reconstruction points 25) and green (greenreconstruction points 27) “checker board” array of FIG. 7, this minimumboundary condition area may lead to both added calculation burden andspreading the data across six sample 74 points.

The alternative effective output sample area 124 arrangement 31 of FIG.9 may be utilized for some applications or situations. For example,where the sample points 74 of FIG. 15 are on the same spatial resolutiongrid and co-incident with the red (red reconstruction points 25) andgreen (green reconstruction points 27) “checker board” array of FIG. 7,or where the relationship between input sample areas 74 and outputsample areas is as shown in FIG. 20 the calculations are simpler. In theeven columns, the formula for calculating the blue output sample points23 is identical to the formula developed above for FIG. 17. In the oddcolumns the calculation for FIG. 20 is as follows:V _(out)(C _(x+) _(—) R _(y) _(—) )=0.25_(—) V _(in)(C _(x) R_(y))+0.25_(—) V _(in)(C _(x+1) R _(y))+0.25_(—) V _(in)(C _(x) R_(y−1)) +0.25_(—) V _(in)(C _(x+1) R _(y−1)).

As usual, the above calculations for FIGS. 19 and 20 are done for thegeneral case of the central sample area 124. The calculations at theedges will require modifications to the transform formulae orassumptions about the values of sample points 74 off the edge of thescreen, as described above.

Turning now to FIG. 21, an array 104 of sample points 122 and theireffective sample areas 120 for a prior art pixel data format isillustrated. FIG. 21 illustrates the red, green, and blue values thatare on an equal spatial resolution grid and co-incident, however, it hasa different image size than the image size illustrated in FIG. 15.

FIG. 22 illustrates an array 106 of sample points 122 and theireffective sample areas 120 of FIG. 21 overlaid on the red color planesampling areas 50, 52, 53, and 54 of FIG. 13. The sample points 122 ofFIG. 21 are not on the same spatial resolution grid, nor co-incidentwith the red (red reconstruction points 25, 35) and green (greenreconstruction points 27, 37) “checker board” array of FIG. 7 or 11,respectively.

In this arrangement of FIG. 22, a single simplistic transform equationcalculation for each output sample 35 is not allowed. However,generalizing the method used to generate each of the calculation basedon the proportional area covered is both possible and practical. This istrue if for any given ratio of input to output image, especially thosethat are common in the industry as standards, there will be least commondenominator ratios that will result in the image transform being arepeating pattern of cells. Further reductions in complexity occur dueto symmetry, as demonstrated above with the input and output arraysbeing coincident. When combined, the repeating three-color sample points122 and symmetry results in a reduction of the number of sets of uniquecoefficients to a more manageable level.

For example, the commercial standard display color image format called“VGA” (which used to stand for Video Graphics Adapter but now it simplymeans 640×480) has 640 columns and 480 rows. This format needs to bere-sampled or scaled to be displayed onto a panel of the arrangementshown in FIG. 10, which has 400 red sub-pixels 34 and 400 greensub-pixels 36 across (for a total of 800 sub-pixels across) and 600total sub-pixels 34 and 36 down. This results in an input pixel tooutput sub-pixel ratio of 4 to 5. The transfer equations for each redsub pixel 34 and each green sub-pixel 36 can be calculated from thefractional coverage of the input sample areas 120 of FIG. 22 by thesample output areas 52. This procedure is similar to the development ofthe transfer equations for FIG. 18, except the transfer equations seemto be different for every single output sample point 35. Fortunately ifyou proceed to calculate all these transfer equations a pattern emerges.The same five transfer equations repeat over and over across a row, andanother pattern of five equations repeat down each column. The endresult is only 5×5 or twenty-five unique sets of equations for this casewith a pixel to sub-pixel ratio of 4:5. This reduces the uniquecalculations to twenty-five sets of coefficients. In these coefficients,other patterns of symmetries can be found which reduce the total numberof coefficient sets down to only six unique sets. The same procedurewill produce an identical set of coefficients for the arrangement 20 ofFIG. 6.

The following is an example describing how the coefficients arecalculated, using the geometric method described above. FIG. 32illustrates a single 5×5 repeat cell 202 from the example above ofconverting a 640×480 VGA format image to a PenTile matrix with 800×600total red and green sub pixels. Each of the square sub-pixels 204bounded by solid lines 206 indicates the location of a red or green subpixel that must have a set of coefficients calculated. This wouldrequire 25 sets of coefficients to be calculated, were it not forsymmetry. FIG. 32 will be discussed in more detail later.

FIG. 33 illustrates the symmetry in the coefficients. If thecoefficients are written down in the common matrix form for filterkernels as used in the industry, the filter kernel for sub-pixel 216would be a mirror image, flipped left-to-right of the kernel forsub-pixel 218. This is true for all the sub pixels on the right side ofsymmetry line 220, each having a filter kernel that is the mirror imageof the filter kernel of an opposing sub-pixel. In addition, sub-pixel222 has a filter kernel that is a mirror image, flipped top-to-bottom ofthe filter kernel for sub-pixel 218. This is also true of all the otherfilter kernels below symmetry line 224; each is the mirror image of anopposing sub-pixel filter. Finally, the filter kernel for sub-pixel 226is a mirror image, flipped on a diagonal, of the filter for sub-pixel228. This is true for all the sub-pixels on the upper right of symmetryline 230: their filters are diagonal mirror images of the filters of thediagonal opposing sub-pixel filter. Finally, the filter kernels on thediagonal are internally diagonally symmetrical, with identicalcoefficient values on diagonally opposite sides of symmetry line 230. Anexample of a complete set of filter kernels is provided further hereinto demonstrate all these symmetries in the filter kernels. The onlyfilters that need to be calculated are the shaded ones, sub-pixels 218,228, 232, 234, 236, and 238. In this case, with a repeat cell size of 5,the minimum number of filters needed is only six. The remaining filterscan be determined by flipping the 6 calculated filters on differentaxes. Whenever the size of a repeat cell is odd, the formula fordetermining the minimum number of filters is:

${Nfilts} = \frac{\frac{P + 1}{2} \cdot ( {1 + \frac{P + 1}{2}} )}{2}$where P is the odd width and height of the repeat cell, and Nfilts isthe minimum number of filters required.

FIG. 34 illustrates an example of the case where the repeat cell size iseven. The only filters that need to be calculated are the shaded ones,sub-pixels 240, 242, and 244. In this case with a repeat cell size of 4only three filters must be calculated. Whenever the size of the repeatcell is even, the general formula for determining the minimum number offilters is:

${Neven} = \frac{\frac{P}{2} \cdot ( {1 + \frac{P}{2}} )}{2}$where P is the even width and height of the repeat cell, and Neven isthe minimum number of filters required.

Returning to FIG. 32, the rendering boundary 208 for the centralsub-pixel encloses an area 210 that overlaps four of the original inputpixel sample areas 212. Each of these overlapping areas is equal, andtheir coefficients must add up to one, so each of them is ¼ or 0.25.These are the coefficients for sub-pixel 238 in FIG. 33 and the2.times.2 filter kernel for this case would be:

1/4 1/4 1/4 1/4

The coefficients for sub-pixel 218 in FIG. 33 are developed in FIG. 35.This sub-pixel 218 is bounded by a rendering area 246 that overlaps fiveof the surrounding input pixel sample areas 248. Although this sub-pixelis in the upper left corner of a repeat cell, it is assumed for the sakeof calculation that there is always another repeat cell past the edgewith additional sample areas 248 to overlap. These calculations arecompleted for the general case and the edges of the display will behandled with a different method as described above. Because renderingarea 246 crosses three sample areas 248 horizontally and threevertically, a 3×3 filter kernel will be necessary to hold all thecoefficients. The coefficients are calculated as described before: thearea of each input sample area covered by rendering area 246 is measuredand then divided by the total area of rendering area 246. Rendering area246 does not overlap the upper left, upper right, lower left, or lowerright sample areas 248 at all so their coefficients are zero. Renderingarea 246 overlaps the upper center and middle left sample areas 248 by⅛^(th) of the total area of rendering area 246, so their coefficientsare ⅛^(th). Rendering area 246 overlaps the center sample area 248 bythe greatest proportion, which is 11/16^(ths). Finally rendering area246 overlaps the middle right and bottom center sample areas 248 by thesmallest amount of 1/32^(nd). Putting these all in order results in thefollowing coefficient filter kernel:

0 1/8 0 1/8 11/16 1/32 0  1/32 0

Sub-pixel 232 from FIG. 33 is illustrated in FIG. 36 with its renderingarea 250 overlapping five sample areas 252. As before, the portions ofthe area of rendering area 250 that overlap each of the sample areas 252are calculated and divided by the area of rendering area 250. In thiscase, only a 3×2 filter kernel would be necessary to hold all thecoefficients, but for consistency a 3×3 will be used. The filter kernelfor FIG. 36 would be:

1/64 17/64 0 7/64 37/64 2/64 0 0 0

Sub-pixel 234 from FIG. 33 is illustrated in FIG. 37 with its renderingarea 254 overlapping sample areas 256. The coefficient calculation forthis would result in the following kernel:

 4/64 14/64 0 14/64 32/64 0 0 0 0

Sub-pixel 228 from FIG. 33 is illustrated in FIG. 38 with its renderingarea 258 overlapping sample areas 260. The coefficient calculations forthis case would result in the following kernel:

4/64 27/64 1/64 4/64 27/64 1/64 0 0 0

Finally, sub-pixel 236 from FIG. 33 is illustrated in FIG. 39 with itsrendering area 262 overlapping sample areas 264. The coefficientcalculations for this case would result in the following kernel:

9/64 23/64 0 9/64 23/64 0 0 0 0

This concludes all the minimum number of calculations necessary for theexample with a pixel to sub-pixel ratio of 4:5. All the rest of the 25coefficient sets can be constructed by flipping the above six filterkernels on different axes, as described with FIG. 33.

For the purpose of scaling the filter kernels must always sum to one orthey will effect the brightness of the output image. This is true of allsix filter kernels above. However, if the kernels were actually used inthis form the coefficients values would all be fractions and requirefloating point arithmetic. It is common in the industry to multiply allthe coefficients by some value that converts them all to integers. Theninteger arithmetic can be used to multiply input sample values by thefilter kernel coefficients, as long as the total is divided by the samevalue later. Examining the filter kernel above, it appears that 64 wouldbe a good number to multiply all the coefficients by. This would resultin the following filter kernel for sub-pixel 218 from FIG. 35:

0 8 0 8 44 2 0 2 0 (divided by 64.)

All the other filter kernels in this case can be similarly modified toconvert them to integers for ease of calculation. It is especiallyconvenient when the divisor is a power of two, which it is in this case.A division by a power of two cam be completed rapidly in software orhardware by shifting the result to the right. In this case, a shift tothe right by 6 bits will divide by 64.

In contrast, a commercial standard display color image format called XGA(which used to stand for Xtended Graphics Adapter but now simply means1024×768) has 1024 columns and 768 rows. This format can be scaled todisplay on an arrangement 38 of FIG. 10 that has 1600 by 1200 red andgreen emitters 34 and 36 and 800 by 600 blue emitters 32. The scaling orre-sampling ratio of this configuration is 16 to 25, which results in625 unique sets of coefficients. Using symmetry in the coefficientsreduces the number to a more reasonable 91 sets. But even this smallernumber of filters would be tedious to do by hand, as described above.Instead a computer program (a machine readable medium) can automate thistask using a machine (e.g., a computer) and produce the sets ofcoefficients quickly. In practice, this program is used once to generatea table of filter kernels for any given ratio. Then that table is usedby scaling/rendering software or burned into the ROM (Read Only Memory)of hardware that implements scaling and sub-pixel rendering.

The first step that the filter generating program must complete iscalculating the scaling ratio and the size of the repeat cell. This iscompleted by dividing the number of input pixels and the number ofoutput sub-pixels by their GCD (Greatest Common Denominator). This canalso be accomplished in a small doubly nested loop. The outer loop teststhe two numbers against a series of prime numbers. This loop should rununtil it has tested primes as high as the square root of the smaller ofthe two pixel counts. In practice with typical screen sizes it shouldnever be necessary to test against primes larger than 41. Conversely,since this algorithm is intended for generating filter kernels “offline”ahead of time, the outer loop could simply run for all numbers from 2 tosome ridiculously large number, primes and non-primes. This may bewasteful of CPU time, because it would do more tests than necessary, butthe code would only be run once for a particular combination of inputand output screen sizes.

An inner loop tests the two pixel counts against the current prime. Ifboth counts are evenly divisible by the prime, then they are bothdivided by that prime and the inner loop continues until it is notpossible to divide one of the two numbers by that prime again. When theouter loop terminates, the remaining small numbers will have effectivelybeen divided by the GCD. The two numbers will be the “scale ratio” ofthe two pixel counts.

-   -   Some typical values:

320:640 becomes 1:2 384:480 becomes 4:5 512:640 becomes 4:5 480:768becomes 5:8 640:1024 becomes 5:8

These ratios will be referred to as the pixel to sub-pixel or P:S ratio,where P is the input pixel numerator and S is the sub-pixel denominatorof the ratio. The number of filter kernels needed across or down arepeat cell is S in these ratios. The total number of kernels needed isthe product of the horizontal and vertical S values. In almost all thecommon VGA derived screen sizes the horizontal and vertical repeatpattern sizes will turn out to be identical and the number of filtersrequired will be S². From the table above, a 640×480 image being scaledto a 1024×768 PenTile matrix has a P:S ratio of 5:8 and would require8×8 or 64 different filter kernels (before taking symmetries intoaccount).

In a theoretical environment, fractional values that add up to one areused in a filter kernel. In practice, as mentioned above, filter kernelsare often calculated as integer values with a divisor that is appliedafterwards to normalize the total back to one. It is important to startby calculating the weight values as accurately as possible, so therendering areas can be calculated in a co-ordinate system large enoughto assure all the calculations are integers. Experience has shown thatthe correct co-ordinate system to use in image scaling situations is onewhere the size of an input pixel is equal to the number of output subpixels across a repeat cell, which makes the size of an output pixelequal the number of input pixels across a repeat cell. This iscounter-intuitive and seems backwards. For example, in the case ofscaling 512 input pixels to 640 with a 4:5 P:S ratio, you can plot theinput pixels on graph paper as 5×5 squares and the output pixels on topof them as 4×4 squares. This is the smallest scale at which both pixelscan be drawn, while keeping all the numbers integers. In thisco-ordinate system, the area of the diamond shaped rendering areascentered over the output sub-pixels is always equal to twice the area ofan output pixel or 2*P². This is the minimum integer value that can beused as the denominator of filter weight values.

Unfortunately, as the diamond falls across several input pixels, it canbe chopped into triangular shapes. The area of a triangle is the widthtimes the height divided by two and this can result in non-integervalues again. Calculating twice the area solves this problem, so theprogram calculates areas multiplied by two. This makes the minimumuseful integer filter denominator equal to 4*P².

Next it is necessary to decide how large each filter kernel must be. Inthe example completed by hand above, some of the filter kernels were2×2, some were 3×2 and others were 3×3. The relative sizes of the inputand output pixels, and how the diamond shaped rendering areas can crosseach other, determine the maximum filter kernel size needed. Whenscaling images from sources that have more than two output sub-pixelsacross for each input pixel (e.g., 100:201 or 1:3), a 2×2 filter kernelbecomes possible. This would require less hardware to implement. Furtherthe image quality is better than prior art scaling since the resultingimage captures the “square-ness” of the implied target pixel, retainingspatial frequencies as best as is possible, represented by the sharpedges of many flat panel displays. These spatial frequencies are used byfont and icon designers to improve the apparent resolution, cheating theNyquist limit well known in the art. Prior art scaling algorithms eitherlimited the scaled spatial frequencies to the Nyquist limit usinginterpolation, or kept the sharpness, but created objectionable phaseerror.

When scaling down there are more input pixels than output sub-pixels. Atany scale factor greater than 1:1 (e.g., 101:100 or 2:1) the filter sizebecomes 4×4 or larger. It will be difficult to convince hardwaremanufacturers to add more line buffers to implement this. However,staying within the range of 1:1 and 1:2 has the advantage that thekernel size stays at a constant 3×3 filter. Fortunately, most of thecases that will have to be implemented in hardware fall within thisrange and it is reasonable to write the program to simply generate 3×3kernels. In some special cases, like the example done above by hand,some of the filter kernels will be smaller than 3×3. In other specialcases, even though it is theoretically possible for the filter to become3×3, it turns out that every filter is only 2×2. However, it is easierto calculate the kernels for the general case and easier to implementhardware with a fixed kernel size.

Finally, calculating the kernel filter weight values is now merely atask of calculating the areas (times two) of the 3×3 input pixels thatintersect the output diamond shapes at each unique (non symmetrical)location in the repeat cell. This is a very straightforward “rendering”task that is well known in the industry. For each filter kernel, 3×3 ornine coefficients are calculated. To calculate each of the coefficients,a vector description of the diamond shaped rendering area is generated.This shape is clipped against the input pixel area edges. Polygonclipping algorithms that are well known in the industry are used.Finally, the area (times two) of the clipped polygon is calculated. Theresulting area is the coefficient for the corresponding cell of thefilter kernel. A sample output from this program is shown below:

Source pixel resolution 1024

Destination sub-pixel resolution 1280

Scaling ratio is 4:5

Filter numbers are all divided by 256

Minimum filters needed (with symmetries): 6

Number of filters generated here (no symmetry): 25

$\begin{matrix}\begin{matrix}0 & 32 & 0 \\32 & 176 & 8 \\0 & 8 & 0\end{matrix} & \begin{matrix}4 & 28 & 0 \\68 & 148 & 0 \\0 & 8 & 0\end{matrix} & \begin{matrix}16 & 16 & 0 \\108 & 108 & 0 \\4 & 4 & 0\end{matrix} & \begin{matrix}28 & 4 & 0 \\148 & 68 & 0 \\8 & 0 & 0\end{matrix} & \begin{matrix}0 & 32 & 0 \\8 & 176 & 32 \\0 & 8 & 0\end{matrix} \\\begin{matrix}4 & 68 & 0 \\28 & 148 & 8 \\0 & 0 & 0\end{matrix} & \begin{matrix}16 & 56 & 0 \\56 & 128 & 0 \\0 & 0 & 0\end{matrix} & \begin{matrix}36 & 36 & 0 \\92 & 92 & 0 \\0 & 0 & 0\end{matrix} & \begin{matrix}56 & 16 & 0 \\128 & 56 & 0 \\0 & 0 & 0\end{matrix} & \begin{matrix}0 & 68 & 4 \\8 & 148 & 28 \\0 & 0 & 0\end{matrix} \\\begin{matrix}16 & 108 & 4 \\16 & 108 & 4 \\0 & 0 & 0\end{matrix} & \begin{matrix}36 & 92 & 0 \\36 & 92 & 0 \\0 & 0 & 0\end{matrix} & \begin{matrix}64 & 64 & 0 \\64 & 64 & 0 \\0 & 0 & 0\end{matrix} & \begin{matrix}92 & 36 & 0 \\92 & 36 & 0 \\0 & 0 & 0\end{matrix} & \begin{matrix}4 & 108 & 16 \\4 & 108 & 16 \\0 & 0 & 0\end{matrix} \\\begin{matrix}28 & 148 & 8 \\4 & 68 & 0 \\0 & 0 & 0\end{matrix} & \begin{matrix}56 & 128 & 0 \\16 & 56 & 0 \\0 & 0 & 0\end{matrix} & \begin{matrix}92 & 92 & 0 \\36 & 36 & 0 \\0 & 0 & 0\end{matrix} & \begin{matrix}128 & 56 & 0 \\56 & 16 & 0 \\0 & 0 & 0\end{matrix} & \begin{matrix}8 & 148 & 28 \\0 & 68 & 4 \\0 & 0 & 0\end{matrix} \\\begin{matrix}0 & 8 & 0 \\32 & 176 & 8 \\0 & 32 & 0\end{matrix} & \begin{matrix}0 & 8 & 0 \\68 & 148 & 0 \\4 & 28 & 0\end{matrix} & \begin{matrix}4 & 4 & 0 \\108 & 108 & 0 \\16 & 16 & 0\end{matrix} & \begin{matrix}8 & 0 & 0 \\148 & 68 & 0 \\28 & 4 & 0\end{matrix} & \begin{matrix}0 & 8 & 0 \\8 & 176 & 32 \\0 & 32 & 0\end{matrix}\end{matrix}$

In the above sample output, all 25 of the filter kernels necessary forthis case are calculated, without taking symmetry into account. Thisallows for the examination of the coefficients and to verify visuallythat there is a horizontal, vertical, and diagonal symmetry in thefilter kernels in these repeat cells. As before, edges and corners ofthe image may be treated uniquely or may be approximated by filling inthe “missing” input sample with the value of either the average of theothers, the most significant single contributor, or black. Each set ofcoefficients is used in a filter kernel, as is well known in the art.Keeping track of the positions and symmetry operators is a task for thesoftware or hardware designer using modulo math techniques, which arealso well known in the art. The task of generating the coefficients is asimple matter of calculating the proportional overlap areas of the inputsample area 120 to output sample area 52 for each sample correspondingoutput sample point 35, using means known in the art

FIG. 23 illustrates an array 108 of sample points 122 and theireffective sample areas 120 of FIG. 21 overlaid on the blue color planesampling areas 44 of FIG. 12, in which the sample points 122 of FIG. 21are not on the same spatial resolution grid, nor co-incident with thered and green “checker board” array of FIG. 11. The method of generatingthe transform equation calculations proceed as described earlier. First,the size of the repeating array of three-color pixel elements isdetermined, next the minimum number of unique coefficients isdetermined, and then the values of those coefficients by theproportional overlap of input sample areas 120 to output sample areas 44for each corresponding output sample point 46 is determined. Each ofthese values are applied to the transform equation. The array ofrepeating three-color pixel elements and resulting number ofcoefficients is the same number as that determined for the red and greenplanes.

FIG. 24 illustrates the array 110 of sample points and their effectivesample areas of FIG. 21 overlaid on the blue color plane sampling areas123 of FIG. 8, in which the sample points 122 of FIG. 21 are not on thesame spatial resolution grid nor co-incident with the red (redreconstruction points 35) and green (green reconstruction points 37)“checker board” array of FIG. 11. The method of generating the transformequation calculations proceeds as described above. First, the size ofthe repeating array of three-color pixel elements is determined. Next,the minimum number of unique coefficients is determined, and then thevalues of those coefficients by the proportional overlap of input sampleareas 120 to output sample areas 123 for each corresponding outputsample point 23 is determined. Each of these values are applied to thetransform equation.

The preceding has examined the RGB format for CRT. A conventional RGBflat panel display arrangement 10 has red 4, green 6, and blue 2emitters arranged in a three-color pixel element 8, as in prior artFIG. 1. To project an image formatted according to this arrangement ontothe three-color pixel element illustrated in FIG. 6 or in FIG. 10, thereconstruction points must be determined. The placement of the red,green, and blue reconstruction points is illustrated in the arrangement12 presented in prior art FIG. 2. The red, green, and bluereconstruction points are not coincident with each other; there is ahorizontal displacement. According to prior art disclosed byBenzschawel, et al. in U.S. Pat. No. 5,341,153, and later by Hill, etal. in U.S. Pat. No. 6,188,385, these locations are used as samplepoints 3, 5, and 7 with sample areas, as shown in prior art FIG. 3 forthe red color plane 14, in prior art FIG. 4 for the blue color plane 16,and prior art FIG. 5 for the green color plane 18.

A transform equation calculation can be generated from the prior artarrangements presented in FIGS. 3, 4, and 5 from the methods disclosedherein. The methods that have been outlined above can be utilized bycalculating the coefficients for the transform equations, or filterkernels, for each output sample point of the chosen prior artarrangement. FIG. 25 illustrates the effective sample area 125 of thered color plane of FIG. 3 overlaid on the red color plane sampling areas52 of FIG. 13, where the arrangement of red emitters 35 in FIG. 25 hasthe same pixel level (repeat unit) resolution as the arrangement in FIG.6 and FIG. 10. The method of generating the transform equationcalculations proceeds as described above. First, the size of therepeating array of three-color pixel elements is determined. The minimumnumber of unique coefficients are then determined by noting the symmetry(in this case: 2). Then, then the values of those coefficients, by theproportional overlap of input sample areas 125 to output sample areas 52for each corresponding output sample point 35 is determined. Each ofthese values are applied to the transform equation. The calculation forthe resampling of the green color plane, as illustrated in FIG. 4,proceeds in a similar manner, but the output sample array is rotated by180° and the green input sample areas 127 are offset. FIG. 26illustrates the effective sample areas 127 of the blue color plane ofprior art FIG. 4 overlaid on the blue color plane sampling areas 123 ofFIG. 8.

FIG. 40 illustrates an example for blue that corresponds to the red andgreen example in FIG. 32. Sample area 266 in FIG. 40 is a square areainstead of a diamond area as shown in the red and green example of FIG.32. The number of original input pixel boundaries 272 shown in dashedlines is the same, but there are fewer blue output pixel boundaries 274.The coefficients are calculated as described before; the area of eachinput sample area 268 (bounded by dashed lines) that is covered bysample area 266 is measured and then divided by the total area of samplearea 266. In this example, sample area 266 equally overlaps four of theoriginal input pixel areas 268, resulting in a 2×2 filter kernel withfour coefficients of ¼. The eight other blue output pixel areas 270(bounded by solid lines) and their geometrical intersections withoriginal input pixel areas 268 can be seen in FIG. 40. The symmetricalrelationships of the resulting filters can be observed in thesymmetrical arrangements of original pixel boundaries 274 in each outputpixel area 270

In more complicated cases, a computer program is used to generate bluefilter kernels. This program turns out to be very similar to the programfor generating red and green filter kernels. The blue sub-pixel samplepoints 33 in FIG. 11 are twice as far apart as the red and green samplepoints 35, 37, suggesting that the blue rendering areas will be twice aswide. However, the rendering areas for red and green are diamond shapedand are thus twice as wide as the spacing between the sample points.This makes the rendering areas of red and green and blue the same widthand height which results in several convenient numbers; the size of thefilter kernels for blue will be identical to the ones for red and green.Also the repeat cell size for blue will generally be identical to therepeat cell size for red and green. Because the blue sub-pixel samplepoints 33 are spaced twice as far apart, the P:S (pixel to sub-pixel)ratio is doubled. For example, a ratio of 2:3 for red becomes 4:3 forblue. However, it is the S number in this ratio that determines therepeat cell size and that is not changed by doubling. However, if thedenominator happens to be divisible by two, there is an additionaloptimization that can be done. In that case, the two numbers for bluecan be divided by an additional power of two. For example, if the redand green P:S ratio is 3:4, then the blue ratio would be 6:4 which canbe simplified to 3:2. This means that in these (even) cases the bluerepeat cell size can be cut in half and the total number of filterkernels required will be one quarter that of red and green. Conversely,for simplicity of algorithms or hardware designs, it is possible toleave the blue repeat cell size identical to that of red and green. Theresulting set of filter kernels will have duplicates (quadruplicates,actually) but will work identically to the red and green set of filterkernels.

Therefore, the only modifications necessary to take the red and greenfilter kernel program and make it generate blue filter kernels was todouble the numerator of the P:S ratio and change the rendering area to asquare instead of a diamond.

Now consider the arrangement 20 of FIG. 6 and the blue sample areas 124of FIG. 9. This is similar to the previous example in that the bluesample areas 124 are squares. However, because every other column ofthem are staggered half of their height up or down, the calculations arecomplicated. At first glance it seems that the repeat cell size will bedoubled horizontally. However the following procedure has beendiscovered to produce the correct filter kernels:

1) Generate a repeat cell set of filter kernels as if the blue samplepoints are not staggered, as described above. Label the columns and rowsof the table of filters for the repeat cell with numbers starting withzero and ending at the repeat cell size minus one.

2) On the even columns in the output image, the filters in the repeatcell are correct as is. The modulo in the repeat cell size of the outputY co-ordinate selects which row of the filter kernel set to use, themodulo in the repeat cell size of the X co-ordinate selects a column andtells which filter in the Y selected row to use.

3) On the odd output columns, subtract one from the Y co-ordinate beforetaking the modulo of it (in the repeat cell size). The X co-ordinate istreated the same as the even columns. This will pick a filter kernelthat is correct for the staggered case of FIG. 9.

In some cases, it is possible to perform the modulo calculations inadvance and pre-stagger the table of filter kernels. Unfortunately thisonly works in the case of a repeat cell with an even number of columns.If the repeat cell has an odd number of columns, the modulo arithmeticchooses the even columns half the time and the odd ones the other halfof the time. Therefore, the calculation of which column to stagger mustbe made at the time that the table is used, not beforehand.

Finally, consider the arrangement 20 of FIG. 6 and the blue samplingareas 123 of FIG. 8. This is similar to the previous case with theadditional complication of hexagonal sample areas. The first stepconcerning these hexagons is how to draw them correctly or generatevector lists of them in a computer program. To be most accurate, thesehexagons must be minimum area hexagons, however they will not be regularhexagons. A geometrical proof can easily be completed to illustrate inFIG. 41 that these hexagon sampling areas 123 of FIG. 8 are ⅛ wider oneach side than the square sampling areas 276. Also, the top and bottomedge of the hexagon sampling areas 123 are ⅛ narrower on each end thanthe top and bottom edge of the square sampling areas 276. Finally, notethat the hexagon sampling areas 123 are the same height as the squaresampling areas 276.

Filter kernels for these hexagonal sampling areas 123 can be generatedin the same geometrical way as was described above, with diamonds forred and green or squares for blue. The rendering areas are simplyhexagons and the area of overlap of these hexagons with the surroundinginput pixels is measured. Unfortunately, when using the slightly widerhexagonal sampling areas 123, the size of the filter kernels sometimesexceeds a 3.times.3 filter, even when staying between the scaling ratiosof 1:1 and 1:2. Analysis shows that if the scaling ratio is between 1:1and 4:5 the kernel size will be 4×3. Between scaling ratios of 4:5 and1:2, the filter kernel size will remain 3×3. (Note that because thehexagonal sampling areas 123 are the same height as the square samplingareas 276 the vertical size of the filter kernels remains the same).

Designing hardware for a wider filter kernel is not as difficult as itis to build hardware to process taller filter kernels, so it is notunreasonable to make 4×3 filters a requirement for hardware basedsub-pixel rendering/scaling systems. However, another solution ispossible. When the scaling ratio is between 1:1 and 4:5 the squaresampling areas 124 of FIG. 9 are used, which results in 3×3 filters.When the scaling ratio is between 4:5 and 1:2, the more accuratehexagonal sampling areas 123 of FIG. 8 are used and 3×3 filters are alsorequired. In this way, the hardware remains simpler and less expensiveto build. The hardware only needs to be built for one size of filterkernel and the algorithm used to build those filters is the only thingthat changes.

Like the square sampling areas of FIG. 9, the hexagonal sampling areasof FIG. 8 are staggered in every other column. Analysis has shown thatthe same method of choosing the filter kernels described above for FIG.9 will work for the hexagonal sampling areas of FIG. 8. Basically thismeans that the coefficients of the filter kernels can be calculated asif the hexagons are not staggered, even though they always are. Thismakes the calculations easier and prevents the table of filter kernelsfrom becoming twice as big.

In the case of the diamond shaped rendering areas of FIGS. 32 through39, the areas were calculated in a co-ordinate system designed to makeall areas integers for ease of calculation. This occasionally resultedin large total areas and filter kernels that had to be divided by largenumbers while in use. Sometimes this resulted in filter kernels thatwere not powers of two, which made the hardware design more difficult.In the case of FIG. 41, the extra width of the hexagonal rendering areas123 will make it necessary to multiply the coefficients of the filterkernels by even larger numbers to make them all integers. In all ofthese cases, it would be better to find a way to limit the size of thedivisor of the filter kernel coefficients. To make the hardware easierto design, it would be advantageous to be able to pick the divisor to bea power of two. For example if all the filter kernels were designed tobe divided by 256, this division operation could be performed by aneight bit right shift operation. Choosing 256 also guarantees that allthe filter kernel coefficients would be 8-bit values that would fit instandard “byte wide” read-only-memories (ROMs). Therefore, the followingprocedure is used to generate filter kernels with a desired divisor.Since the preferred divisor is 256, it will be utilized in the followingprocedure.

1) Calculate the areas for the filter coefficients using floating pointarithmetic. Since this operation is done off-line beforehand, this doesnot increase the cost of the hardware that uses the resulting tables.

2) Divide each coefficient by the known total area of the renderingarea, then multiply by 256. This will make the filter sum to 256 if allarithmetic is done in floating point, but more steps are necessary tobuild integer tables.

3) Do a binary search to find the round off point (between 0.0 and 1.0)that makes the filter total a sum of 256 when converted to integers. Abinary search is a common algorithm well known in the industry. If thissearch succeeds, you are done. A binary search can fail to converge andthis can be detected by testing for the loop running an excessive numberof times.

4) If the binary search fails, find a reasonably large coefficient inthe filter kernel and add or subtract a small number to force the filterto sum to 256.

5) Check the filter for the special case of a single value of 256. Thisvalue will not fit in a table of 8-bit bytes where the largest possiblenumber is 255. In this special case, set the single value to 255 (256−1)and add 1 to one of the surrounding coefficients to guarantee that thefilter still sums to 256.

FIG. 31 illustrates arrangement 40 of effective reconstruction pointsshown in FIG. 11 overlaid on top of array 70 of FIG. 15 in the specialcase when the scaling ratio is one input pixel for each two output subpixels across. In this configuration 200, when the original data has notbeen sub-pixel rendered, the pairs of red emitters in the three colorpixel element 39 (FIG. 10) would be treated as though combined, with arepresented reconstruction point 33 in the center of the three colorpixel element 39. Similarly, the two green emitters in the three-colorpixel element 39 of FIG. 10 are treated as being a single reconstructionpoint 33 in the center of the three-color pixel element 39. The blueemitter is already in the center. Thus, the five emitters can be treatedas though they reconstructed the RGB data format sample points, asthough all three color planes were in the center. This may be consideredthe “Native Mode” of this arrangement of subpixels.

By resampling, via subpixel rendering, an already sub-pixel renderedimage onto another sub-pixelated display with a different arrangement ofsubpixels, much of the improved image quality of the original isretained. According to one embodiment, it is desirable to generate atransform from this sub-pixel rendered image to the arrangementsdisclosed herein. Referring to FIGS. 1, 2, 3, 4, 5, 25, and 26 themethods that have been outlined above will serve, by calculating thecoefficients for the transform filters for each reconstruction point 35,shown in FIG. 25, of the target display arrangement with respect to therightward displaced red input sample 5 of FIG. 3. The blue emitter istreated as indicated above, by calculating the coefficients for thetransform filters for each output sample point of the target displayarrangement with respect to the displaced blue input sample 7 of FIG. 4.

In a case for the green color plane, illustrated in FIG. 5, where theinput data has been sub-pixel rendered, no change need be made from thenon-sub-pixel rendered case since the green data is still centered.

When applications that use subpixel rendered text are includedalong-side non-subpixel rendered graphics and photographs, it would beadvantageous to detect the subpixel rendering and switch on thealternative spatial sampling filter described above, but switch back tothe regular, for that scaling ratio, spatial sampling filter fornon-subpixel rendered areas, also described in the above. To build sucha detector we first must understand what subpixel rendered text lookslike, what its detectable features are, and what sets it apart fromnon-subpixel rendered images. First, the pixels at the edges of blackand white subpixel rendered fonts will not be locally color neutral;That is R≠G. However, over several pixels the color will be neutral;That is R≅G. With non-subpixel rendered images or text, these twoconditions together do not happen. Thus, we have our detector, test forlocal R≠G and R≅G over several pixels.

Since subpixel rendering on an RGB stripe panel is one dimensional,along the horizontal axis, row by row, the test is one dimensional.Shown below is one such test:If R_(x)≠G_(x) andIf R _(x−2) +R _(x−1) +R _(x) +R _(x+1) +R _(x+2) ≅G _(x−2) +G _(x−1) +G_(x) +G _(x+1) +G _(x+2)OrIf R _(x−1) +R _(x) +R _(x+1) +R _(x+2) ≅G _(x−2) +G _(x−1) +G _(x) +G_(x+1)

Then apply alternative spatial filter for sub-pixel rendering input,

Else apply regular spatial filter.

For the case where the text is colored there will be a relationshipbetween the red and green components of the form R_(x)≈aG_(x), where “a”is a constant. For black and white text “a” has the value of one. Thetest can be expanded to detect colored as well as black and white text:If R _(x) ≠aG _(x) andIf R _(x−2) +R _(x−1) +R _(x) +R _(x+1) +R _(x+2) ≅a(G _(x−2) +G _(x−1)+G _(x) +G _(x+1))OrIf R _(x−1) +R _(x) +R _(x+1) +R _(x+2) ≅a(G _(x−2) +G _(x−1) +G _(x) +G_(x+2))

Then apply alternative spatial filter for subpixel rendering input

Else apply regular spatial filter.

R_(x) and G_(x) represent the values of the red and green components atthe “x” pixel column coordinate.

There may be a threshold test to determine if R≅G is close enough. Thevalue of the threshold may be adjusted for best results. The length ofterms, the span of the test may be adjusted for best results, but willgenerally follow the form above.

FIG. 27 illustrates an arrangement of three-color pixel elements in anarray, in three planes, for a display device according to anotherembodiment. FIG. 28 illustrates the arrangement of the blue emitterpixel elements in an array for the device of FIG. 27. FIG. 29illustrates the arrangement of the green emitter pixel elements in anarray for the device of FIG. 27. FIG. 30 illustrates the arrangement ofthe red emitter pixel elements in an array for the device of FIG. 27.This arrangement and layout is useful for projector based displays thatuse three panels, one for each red, green, and blue primary, whichcombine the images of each to project on a screen. The emitterarrangements and shapes match closely to those of FIGS. 8, 13, and 14,which are the sample areas for the arrangement shown in FIG. 6. Thus,the graphics generation, transform equation calculations and dataformats, disclosed herein, for the arrangement of FIG. 6 will also workfor the three-panel arrangement of FIG. 27.

For scaling ratios above approximately 2:3 and higher, the subpixelrendered resampled data set for the PenTile™ matrix arrangements ofsubpixels is more efficient at representing the resulting image. If animage to be stored and/or transmitted is expected to be displayed onto aPenTile™ display and the scaling ratio is 2:3 or higher, it isadvantageous to perform the resampling before storage and/ortransmission to save on memory storage space and/or bandwidth. Such animage that has been resampled is called “prerendered”. This prerenderingthus serves as an effectively lossless compression algorithm.

The present disclosure enables skilled artisans to take most any storedcolor image that is rendered according to source subpixel samples takenaccording to a first spatial distribution of the source subpixels and torender the color image onto a color display having display subpixelsspatially arranged according to a different second spatial distributionof the display subpixels.

While the present disclousure of invention has been described withreference to exemplary embodiment provided herein, it will be understoodby those skilled in the art that various changes may be made andequivalents may be substituted for elements thereof without departingfrom the scope and spirit of the disclosure. In addition, manymodifications may be made to adapt a particular situation or material tothe present teachings without departing from the scope thereof.Therefore, it is intended that the disclosure not be limited to theparticular embodiment disclosed herein.

1. A method of selectively providing spatial sampling filtering in adisplay system that processes image data signals representing both imageareas that are sub-pixel rendered areas and image areas that arenon-sub-pixel rendered areas, wherein the selectively provided spatialsampling filtering transforms the image data signals into spatiallyfiltered signals, the method being implemented by the display system andcomprising: detecting sub-pixel rendered areas of the image data signalsby testing portions of the image data signals for satisfaction of afirst condition, the first condition being satisfied when for a firstpixel within one of the tested portions, a first color luminance definedby a corresponding portion of the image data signal for a firstto-be-displayed sub-pixel-emitted color of the first pixel is not equalto a second color luminance defined by a corresponding portion of theimage data signal for a second to-be-displayed, different andsub-pixel-emitted color of the first pixel and further when said secondcolor luminances as averaged out due to respective sub-pixel emissons ofthe first pixel and successively adjacent other pixels in a predefinedneighborhood of the given first pixel are substantially equal to oneanother; turning on, within the display system, a first set of spatialsampling filters for corresponding said subpixel rendered areas inresponse to the detecting of satisfaction of the first condition; andturning on a second set of different spatial sapling filters forcorresponding non-sub-pixel rendered areas in response to failure of thedetection of the satisfaction of the first condition.
 2. The method ofclaim 1, wherein detecting the subpixel rendered areas comprises testingfor satisfaction, as part of the first condition, of the jointconditions defined by:R _(x) ≠G _(x) andR _(x−2) +R _(x−1) +R _(x) +R _(x+1) +R _(x+2) ≅G ⁻² +G _(x−1) +G _(x)+G _(x+1) +G _(x+2), wherein R is a value of red, G is a value of green,and x is a pixel position in a row of pixels.
 3. The method of claim 1,wherein detecting the subpixel rendered areas comprises testing forsatisfaction, as part of the first condition, of the joint conditionsdefined by:R _(x) ≠G _(x) andR _(x−1) +R _(x) +R _(x+1) +R _(x+2) ≅G _(x−2) +G _(x−1) +G _(x) +G_(x+1), wherein R is a value of red, G is a value of green, and x is apixel position in a row of pixels.
 4. The method of claim 1, whereindetecting the subpixel rendered areas comprises testing forsatisfaction, as part of the first condition, of the joint conditionsdefined by:R _(x) ≠aG _(x) andR _(x−2) +R _(x−1) +R _(x) +R _(x+1) +R _(x+2) ≅a(G ⁻² +G _(x−1) +G _(x)+G _(x+1)), wherein R is a value of red, G is a value of green, a is apredefined constant, and x is a pixel position in a row of pixels. 5.The method of claim 1, wherein detecting the subpixel rendered areascomprises testing for satisfaction, as part of the first condition, ofthe joint conditions defined by: R_(x)≠aG_(x) andR_(x−1)+R_(x)+R_(x+1)+R_(x+2)≅a(G_(x−2)+G_(x−1)+G_(x)+G_(x+1)), whereinR is a value of red, G is a value of green, a is a predefined constant,and x is a pixel position in a row of pixels.
 6. The method of claim 1,wherein the image data signals are supplied to the display system andthe display system responsively displays a corresponding image on acorresponding display having pixels, which image is defined at least inpart by said turning on of the first set of spatial sampling filters. 7.The method of claim 6, wherein each pixel of the display includes one ormore first sub-pixel areas the emit the first sub-pixel-emitted colorand one or more second sub-pixel areas that emit the secondsub-pixel-emitted color.
 8. The method of claim 1, wherein saidconditional turning on of the first set of spatial sampling filters andsaid conditional turning on of the second set of spatial samplingfilters occurs prior to display by the display system of an imagedefined by the image data signals representing both image areas that aresub-pixel rendered areas and image areas that are non-pixel renderedareas.